DEVELOPMENT OF LEVEL 2 CALIBRATION AND VALIDATION PLANS FOR GOES-R; 


Introduction 


Calibration and Validation (Cal/Val) plans for Geostationary Operational Environmental Satellite 
version R (GOES-R) Level 2 (L2) products were documented via Resource, Implementation, and 
Management Plans (RIMPs) for all of the official L2 products required from the GOES-R Advanced 
Baseline Imager (ABI). In 2015 the GOES-R program decided to replace the typical Cal/Val plans with 
RIMPs that covered, for a given L2 product, what was required from that product, how it would be 
validated, and what tools would be used to do so. Similar to Level 1b products, the intent was to cover 
the full spectrum of planning required for the Cal/Val of L2 ABI products. Instead of focusing on step- 
by-step procedures, the RIMPs concentrated on the criteria for each stage of the validation process 
(Beta, Provisional, and Full Validation) and the many elements required to prove when each stage was 
reached. 


Approach 


The creation of the L2 RIMPs was a complex and dynamic process. The suite of L2 products required 
18 separate L2 RIMPs guided by 4 lead authors who interacted with 12 different science leads from 
Satellite Applications and Research (STAR) along with GOES-R program personnel. Each L2 RIMP 
had to accommodate a fixed GOES-R validation schedule and GOES-R requirements. Furthermore 
there were conflicting documents on exactly what the standards were for validation, and hundreds of 
pre-existing Post Launch Product Tests (PLPTs) that also were to be addressed within the RIMPs. The 
L2 RIMPs were standardized to insure critical content such as the appropriate criteria for each stage, 
tools, schedule, processes, and the description of pre-existing Validation Events (VEs) were handled in 
a consistent manner. 


Initial efforts determined the scope of the documents, and afterwards organized, coordinated, and 
developed the L2 RIMPs. Discrepancies were identified between the Functional and Performance 
Specification (F&PS) and the Product User’s Guide (PUG), and measures to be used to validate each 
product were successfully established. Risks were identified throughout the development of the RIMPs 
and passed forward to the appropriate review boards. Interactions between the L2 products Cal/Val 
efforts had to be accounted for, especially in the case of the Clear Sky Mask (CSM), which is a 
necessary input to many L2 products. The CSM coordination and development took particular care. 


Contents 


Each L2 RIMP contained the following required content: 

Validation Overview 

Schedule of Events 

Roles and Responsibilities 

Tools 

Analysis Methods 

Output Artifacts 

Pre-Launch 

References 

Appendices: Validation Events, Validation Reference Data, List of Tools, Acronyms 


Cloud and Moisture Imagery 


Cloud and Moisture Imagery (CMI) was a special case, and followed the L1b schedule. Anticipation 
for GOES-R Imagery is high, and numerous pre-launch efforts have focused on being able to exploit 
this Imagery as soon as it becomes available (e.g. Greenwald et al [1]). For CMI, the only L2 Key 
Performance Parameter (KPP), a key challenge was the predominance of non-quantitative criteria. 
While certain aspects of CMI are quantitative, such as properly mapping the radiances from L1b, the 
primary purpose of CMI is the human identification of critical phenomena from sand storms to 
hurricanes to snow cover. Different users will search for different atmospheric and ground features 
depending on their location and event of interest, and the CMI products must accommodate them all. 
Hence the CMI RIMP described an extensive set of display tools that covered the breadth of interests 
across the user community. The CMI RIMP also had to accommodate the overarching schedule of 
when data would first be made available to users, since they could only contribute after such access 
occurred. This coordination was accomplished as part of the CMI RIMP development. 


Clear Sky Mask (CSM) 
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Schedule of the Clear Sky Mask (CSM) validation compared to the overarching schedule of L2 product validation. Large arrows represent the L2 
schedule, thinner arrows are for the CSM. The CSM completes the provisional stage when the other L2 products complete beta. 


The CSM was an exceptional challenge since it impacted numerous other L2 products. Such challenges with cloud mask verification have 
occurred with other programs such as the Suomi National Polar-orbiting Partnership (SNPP) and described in Kopp et al [2]. The Figure above 
shows the overall L2 validation schedule, and the completion dates set for the CSM. The CSM schedule had to account for all downstream 
dependencies, since all of the L2 products had to meet the schedule reflected in Figure 1. None of the dependent L2 products could finish 
validation without an acceptable CSM. Therefore the CSM completion date through Full Validation is a good 200 days before the end of formal 
validation activities. Each dependent product both influenced the CSM completion dates, and likewise were written to correspond with the 
expected availability, and accuracy, of the CSM. Therefore the resulting 18 L2 RIMPs are properly correlated with CSM progress and planning, 
with margin in case of unexpected events. 


The CSM was the initial product in a sequence of cloud products, and RIMPs were required for each of the cloud products. The sequence of the 
entire cloud processing chain is shown below. The cloud product RIMPs were broken down based on this chain. Three cloud RIMPs cover nine 
different cloud products, which were grouped together based on their position in the chain and the similarity of the tools employed to prove 
validation. The exception is cloud phase, whose characteristics and validation techniques are more in line with those used for cloud optical 
parameters. The Cloud Optical Microphysical Properties (COMP) algorithms are significantly different for day and night, and they are treated 
as separate entities (DCOMP — Davtime. NCOMP — Nighttime) in the RIMPs. This anvroach successfullv incornorated all of the cloud products 
without undue duplication of ma 
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Simple schematic of the cloud product algorithm chain for GOES-R. The associated cloud RIMPs are below the algorithm/product, and the 
products were combined consistent with the algorithms that product them. 
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Logistics for the remaining L2 RIMPs 


Similar ideas were employed for land and aerosol products, though for these products each individual product 
retained its own RIMP. Since these algorithms did not run in a chain, it was appropriate to retain individual 
RIMPs. Volcanic ash was a more difficult case, since ash events vary widely over large time scales, and there is 
no guarantee of a significant ash event during the official Cal/Val period. The RIMP accounts for this possibility 
and the RIMP includes options to quantitatively show volcanic ash product performance even if no significant 
event occurs. Each RIMP had to account for upstream product dependencies, which are depicted in the Figure 
below. 
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Although each L2 RIMP followed the same format, additional considerations were necessary for Derived 
Motion Winds (DMW) and Hurricane Intensity Estimation (HIE). In the case of DMW, it contained more than 
triple the number of PLPTs compared to any other L2 product. This proved unwieldly, and only after extensive 
discussion was a consolidation approach agreed to by all parties. The result retained all of the necessary and 
applicable PLPTs while avoiding an unduly lengthy and repetitive RIMP. For HIE the challenge was in 
describing correct quantitative criteria for what is an estimate by design. Nevertheless quantitative values were 
present in requirement documents and were incorporated into the HIE RIMP to provide an effective standard by 
which the various stages of validation may be met. 


Soundings 


Conclusions 


GOES-R, now GOES-16, successfully launched on 19 November 2016. GOES-16 reached its testing position 
(e.g. GOES-Central) on 5 December and validation procedures have passed the beta stage. The L2 RIMPs 
successfully guided the validation process through the beta validation on time, and they continue to be the 
foundation from which GOES-16 is accomplishing the L2 validation process. 
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